In recent years, advances in computer and network technology have led to the growth of networks providing fast and inexpensive access to information resources throughout the world. Networked computers are used at work, at home, and at school to share information and other content. Businesses use computer networks to deliver software, content, and services as well as to advertise and offer goods and services for sale. Consumers and businesses may select and purchase a variety of goods and services on-line. For example, books, clothing, electronics, and automobiles can be purchased on-line through vendor web sites. Likewise, financial services (such as stock trading, banking, and portfolio management), travel services, and news and information services, among others, are available on-line. Equally important, but perhaps not as evident to consumers, are a variety of on-line network services that support on-line businesses, such as transaction processors, security services, and e-mail service providers. Regardless of the type of goods and services provided, all on-line business must be concerned with the performance of their network operations. E-businesses unable to provide consistent, high performance are not likely to survive.
Individual businesses or entities that provide network applications have different network performance concerns. For example, a company may hire a Web service provider to provide its on-line services. The company may wish to conduct periodic audits to ensure that the Web service provider is fulfilling its obligation to provide satisfactory service. Similarly, the Web service provider may wish to validate the level of service quality it is providing and identify any problems that may exist or that may arise.
Network management tools can be used to collect data from various locations and at various times. For example, the tools may collect connect times, download times for individual pages, domain name server (DNS) look-up times, error messages, link octet rate, application octet rate, and link utilization for a server among other things. In fact, network management tools may monitor tens or hundreds of parameters for a given website. Data from the monitoring may be organized into data records. The streams of data records generated by the network management tools may be analyzed to evaluate network performance and detect and diagnose problems. While the description above is primarily directed to network applications, it should be appreciated that data streams can be generated by a variety of detectors, sensors, or other sources and that analysis of those data streams may also be desired.
In conventional automated analysis systems, the measurements are analyzed either as an aggregate (i.e. all intermingled) or as distinct channels based on their origin. For example, the link utilizations from a set of network measurements may be analyzed as a group, in order to determine which, if any, of them are anomalous. The output of this stage, the anomalous links, is useful on its own. However, the sophistication of subsequent analyses is limited since the output of this stage has one key—link identifier in the example.